Identification of Nonstationary Audio Signals Using the FFT, with Application to Analysis-based Synthesis of Sound

In the analysis of sound (for synthesis), digitally sampled audio is processed to extract certain features. The resulting data can be synthesised to reproduce the original sound, or modified before synthesis to musically transform the sound. The analysis primarily uses a harmonic model, which considers a sound to be composed of multiple nonstationary sinusoids. The first stage of analysis is often the Fast Fourier Transform(FFT), where they appear as peaks in the amplitude spectrum. A fundamental assumption when using the FFT is that the signals under investigation are Wide Sense Stationary(WSS); in terms of sinusoids, they are assumed to have constant frequency and amplitude throughout the FFT window. Since musical audio signals are in fact quasi-periodic, this assumption is only a good approximation for short time windows. However the requirement for good frequency resolution necessitates long time windows. Consequently the FFT’s contain artifacts which are due to the nonstationarities of the audio signals. This results in temporal distortion or total mis-detection of sinusoids during analysis, hence reducing synthesised sound quality. This paper presents a technique for extracting nonstationary elements from the FFT, by making use of the artifacts they produce. In particular, linear frequency modulation and exponential amplitude modulation can be determined from the phase distortion that occurs around the spectral peaks in the FFT. Results are presented for simulated data and real audio examples.